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(g) System and methods for file management 

(g) A computer system (100) has ooncunrently 
shared objecte or resources (333) and includes 
a multi-user database management system 
(150) having infonnatlon tables (161, 162, 163) 
and related objects stored in shared directories 
on a tae server (180). A plurality of lock types. 
Including diredory lock, full lock, write lock, 
prevent fufl lock, and prevent write lock, are 
provkled for controlling concurrent access. 

Methods (700) are described for managing 
locks by creating a special lock file (350) for 
each shared directory that is accessed. The lock 
file stores at least one logical lock file (400) 
having locking or concunrency Information 
specrTic to a family of related members. The 
logical lock file itself includes a plurality of 
entries (430) for specifying concurrency infor- 
mation of associated famOy memt)ere. A shared 
object or resource is accessed according to the 
infonmation retrieved from the corresponding 
logical lock file, data retrieval being improved 
by reading an amount of data equal to a previ- 
ously memorized file size plus a predictive 
anxHint 
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The present invention relates generally to file management in data processing 
particularly, is applicable to accessing shared resources In a muiti-user environment, such as a computer net- 

Compters are a powerful tool for the acquisition and processing of information. Computerized databases, 
which can be regarded as a kind of electronic filing cabinet or repository for collecUng 'f"P'^"^fl*'^^^f; 
Tre particularly adept at processing vast amounts of information. As such, these systems serve to mamtam 
infbrmation in database files or tables and make that information avaflabie on demand. 

T^ween the actual physical database rtself (i.e.. the data actually stored on a storage device) and the 
users of the system, a database management system or DBMS is provided as a software cushion or layer, in 
essence the DBMS shields the database userfrom knowing or even caring about underlying hadware-level 
Sis -fypicaliy. all r^uests ftom usere for access to the data are processed by the DBMS For e^ple. 
fnf^rmation ^y be added to or removed ftom data f les. information retrieved ftom or "Plated m such f ,Ies^ 
and so forth, all without knowledge of underlying system Implementatton. In this manner, the DBMS provides 
users withaconceptuaiviewofthedatabasethat is removedftomthehartwarelevel. The genera^ 

and operation of a database management system is known in the art See e.g.. Date. C. An Inboduobon to 
Database Systems. Volume I and II, Addison Wesley. 1990. ,. v.Ko««nor 

Of particular interest to the present Im^entton are those informatton processing systems which are oper- 
ative in a shared fashton. i.e.. by multiple users at a given time. A multi-user datetase h"P«emented ona 
ent/server platform is one such system. Typically. Information sharing or connectivity *f»"J^^ 
provided by a network, which comprises several computere connected together as a group. At 'eastone of tte 
PCS functions as a "server." providing network services to "diente- (other computers) on "^tw"",*^ ."^J 
manner, valuable resources, such as programs. Information tobies, memory, disk space, printere. and the like, 
mav be shared by several users. . 

Inherent in any multi-user computing system is a basic conflict between date integnty and concurrency, 
i e the need to let many users access the same date simulteneously. To ensure date integrrty. such a system 
could allow only one user to use a date teble at any given time, but this would be highjr inconvenient to other 
users. On the other hand, the system could allow anyone on a network to use any table at any Such 
unrestricted access, however, would quiddy lead to inconsistencies in the date. The need for insunng data in- 
tegrity, therefore, must be balanced with the need to provide maximum concurrent ^'^J*''^'^^^'^ 
in designing any multi-user applteatton Is dedding how to resoh^e simuttaneous requeste for the same resourc 

^' The need for concurrency control is perhaps most acute in a multnuser datebase system, where informa- 
tion is frequently or even constently being updated by several users. Suppose, for example that two users are 
both executing an applteation that reads a particular value from a datebase. performs a calculation on the va^ 
ue and writes a new value badt to the datebase. If this process begins concurrently, both users will read t|» 
same datebase value, e.g., three. Suppose the calculation is to increment the datebase value by one. After 
both usere have finished, the new value stored in the datebase will be four. However, the correct ^ue d^ired 
is five, since each of the two intended to add one to the value of three. Thus, the concurrent actions of two 
orocesses have interfered, leaving incorred date in the datebase. ^ ^ 

The technique most commonly employed for coordinating processes and controlling access to shared re- 
sources is a -lock" mechanism. Without such a scheme, a second user may update an object, thus providing 
the first user with dd date (as in the above example). With iod«. objeds (e.g.. tables, reports, forms and 
other resources) are restrided in such a way that interference problems are avokJed. This service is most con- 
veniently managed by the network datebase system, with typically some low-level locking mechanism provid- 
ed by the operating system. In this manner, multiple usere may transparently access the same resources in 
the same datebase at the same time, with date integrity foBy maintained. . k- ♦ 

In ite simplest form, loddng an objed prevente other processes or transadfons from accessing that objed 
(or portion thereoO untH the lode is released. In general, the less that is lodced. the greater the potential for 
concurrency: more processes can simulteneously access the datebase without 

locks. If the iodk is too weak, however, date Integrity may still be compromised. On the other hand, a lock which 
is too strong unnecessarily restricte others from accessing the lodced object Thus, providing the nghttype of 
lock for a user's need is an important conskleration. ^ , . * i 

Two types of lodes are fondamentel to concurrency, write lodes and read lodes. Exdusive or write locks 
allow the user of the lock full access to the locked objed. That user hdds sole access to the objed for reading 
and changing. While a write lock is in place, no other user can read or access that objed. Moreover, no other 
user can place a write tode on that objed <i.e.. it has already been locked by andher). A shared or read lock^ 
in contrast, allows multiple users to access an objed but only guarantees the current state of the object The 
lock guarantees that no other user will be granted write access to the objed: dhers may be granted read ao- 
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cess, however. With a read lock, therefore, multiple users can view an object of a database; at the same time, 
the system prevents any changes to that object 

In addition to the locks themselves, there are also interactions between locks, including some locks block- 
ing other locks. For example, a write lock must block a read lock as well as another write lock. A read lock on 
the other hand, blocks attempts to write to the object, but It does not block other read tocks. If a requested 
lock is blocked by another, the requestor must wait until that block is removed. A system may also provkle a 
"time-out" i.e., a pre-set time limit specifying the maximum time one warts for an object to be unlocked If a 
request for a lock fails, the system typically wflt undo or roU back the current transactton. indudino releasino 
other locks held by that process. 

Regardless of the type or hierarchy of locks provided, a system must be able to process locks In real-time 
In particular, many datebase systems (e.g.. relatranal DBMSs) are designed to manage multiple short trans^ 
actions, each occurring on the order effractions of a second. To accommodate these transactions, locks must 
be capable of being held for only short periods of time. All the while, the datebase system must somehow keep 
track of which objecte are locked and what interactions are occurring between the locks. 

Possible approaches for a system include maintaining a teble of objecte which are locked, marking objecte 
which are locked, and storing locked objecte in a special area of memory. Of particular interest to the present 
inventton Is the first approach which employs "lock tebles" for tracking who holds a lock and what kind of lock 
it IS. Commonly, prior art techniques for managing lock files employ an inefficient system open call mechanism. 
To lock a record in a teble. for example, many steps are required; 1) open the lock file. 2) read the entire contents 
of that file. 3) determine If a conf Ifct of locks existe. and. if not, add a record lock to the lock file, 4) write the 
f He out to physical disk, and 5) dose the f He. As this approach requires numerous disk Input/output (I/O) op- 
erattons, a hefty performance penalty te Incurred. Thus, prior art techniques employing lock files in multi-user 
environmente of even modest size must compromise performance. 

The present inventton has been developed whilst attempting to fulfil a need for a multi-user system having 
the advantages of locking, but without the overhead normally assodated with such a mechanism, in particular 
not only has an improved locking system been devised, but also an informatfon file processing system has 
been devised which, whilst oonceived to implement the control of file locking informatton. is applicable generallv 
to system information files. ^ 
Thus, according to one aspect of the invention, there is provided, in a computer system having a memory 
and a storage device, a method for storing and retrieving a plurality of informatton files, the method comprising: 

(a) storing the plurality of information files as a single storage device file which indudes at least one storage 
block, of uniform size, for each of the information files, and each said information fOe having a memorized 
file size when it has already been read; and 

(b) retrieving a previously retrieved information file from the storage device by 

(i) locating a storage block or blocks having the desired information file, and 

(ii) reading from the storage block or blocks an amount of date equal to the previously memorized file 
sbce of the information file plus a predictive amount 

According to a second aspect of the invention, there is provided, in a computer system having a memory 
and a storage device, a method for storing and retrieving a plurality of information files, the method being char- 
40 acterised by: 

(a) storing the plurality of information files as a single disk file, the disk file induding at least one storage 
block for each of the information files, said at least one storage block having a unifomi size, and each 
said information file having a file size; and 

(b) retrieving a desired informatbn file from the storage device by 

45 (i) locating a storage block having the desired information file, and 

(ii) reading from the storage block an amount of date equal to the file size of the information file plus 
a predictive amount. 

According to a third aspect of the invention, there is provided, in a computer system having a plurality of 
resources concurrentiy accessed on a shared storage means, a method for managing access to the resources 
so the method being characterized by: 

grouping related ones of the resources into individual families; 

storing in a physical lock file in the storage means a plurality of logical lock files, each logical lock file 
stonng information for accessing resources dSa single fam8y. said physical lock file having a plurality of uniform 
storage areas each of which stores no more than one logical lock file; and 

accessing each resource of Interest by reading the logical lock file for the family of the resource. 
Other aspecte of the invention are exemplified by the attached daims 

As a general example, one can provided a multiuser computer system, such as a dient/server system hav- 
ing a ffle server connected through a network to one or more diente or work stations, the system induding 
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lesources which may be shared among several useis concurrently. In a partlcuiarembodiment of such a sj^ 
Srr^SiJiS^^^base management system is p«,vlded; this includes ^^^^^ ^^^^^ 

Srectoriee on the f ile server. Associated with each table are other family members, including forms reports. 
au^S^ anS the a plurality of lock types, including directory lock, full lock, write lock, prevent full lock. 
JS^ert wrJe «e IndJded fbr Zdmizing concurrent access whPe minimizing corruphon and data 

"^Methods aiB provMed for managing locks by creating a special lock file for each shared d'ref onr that 
acces^dTp^c»ically. a -physical- lock file for staring various tocking and related concurrency '"["-."^afo" « 
il^^fn thrd^rectory being accessed. In fcirn. the phystoal lockf Be stores at least one logical tockf He hav,^ 
SS^^g hfbrJ«ton specific to a shared table and its related (family) members. The phys«al 'o^ «e mdudes 
a header and a diredtory. The header stores housekeeping informatton. including a -highwater mark" wWch 
to a^irt ciSoStion where new togical lock files may be stored. The directory, in turn, stores entnes 
for referencinq each logical lock file at a particular location. » *- ^ 

tS Si lock f to itself may include a plurality of entri«. for specifying concurrency ^'"^^^^ 
soclatc^ Snily membere. A preferred layout for a logical lock file Includes a header and tock data area. The 
S!f^sSrs^ping Infcrmation^Lock data, on the other hand, stores specif locking informatton as 

'"'';"Z^^*^^hodTacce^^^^ a shared object or resource proceeds as follows. Rret. a request is re- 
ceivi^^ra^to an Object in a diictory. If a physical lock file does not exist for the directory, then one « 
^ted methoi Checks whether a logical lock file exists for the famtty of interest If none exjsts. 

Z^ers2^rd.Tonrheotherhand.anexls«^ 
am retrieved usinq oredlctlve read methods as described hereinafter. 

niTc^icS lo* fie which has been read has been marked as InvalM (i.e.. because t needed to g.o«0. 
t J the nShod proceeds to retrieve the newly allocated (larger) verston of the f «e. At ^P^^^^"- 
frZatten in the looical lock fie may be processed as desired. In particular, access is granted or denied to the 
TJ^J^^n^r^e^onL cSS^rency informatton read. Moreover, the togical lockfile ftself may 
Tr^med. indudlng adding new registretlon entries and deteting oW on^; In this '"^^-"-J^^^- 

back to the storage disk, with the size written being remembered as the current sae. The method loops 

"it^rgrtocjrrrnrrais:'^^^^ 

of logical lockfiles).oneminimlze3d«k I/O (input/output)operetten8.Mo^ 
oft>?e system need only maintainasingle file handletothe single physk»llodcfHe(^oppo8edtc^ 

fevVml system file handles forvarious physical lodcfDes). The net reeulllssubste^^^ 

in a multi-user environment. . j . j ^^^^..^n^ 

Forabetterunderetandingofthe invention andtoshowhowthe same may be earned into effect, reference 

wDI now be made, by way of example, to the accompanying drawings, in which: 

Fia 1 A is a btock diagram of a computer system in which the present invention is operative. 

IB IS a bSS^ dtegram of a database management system (DBMS) which is operative in the system 

""^ta.tc is a diagram illustrating the storage and management of information in the DBMS of Fig 1 B_ 
Fig. 1 D is a block diagram of a multi-user computing emnronment. such as a tocal area network (LAN), m 
which the present invention may be embodied. ^ . 

Flfl 2A Is a teble Illustrating the types and functions of lodes employed by the present inyenbon. 
F^'. 2B is a block diagram llustrating different user operations in a shared environment which are per- 
45 formed on the database tables of the system. ^ . ^. . ^ 

Rg. 2C Is a btock diagram ilustreting the respective lodes whldi are asserted in the shared environment 

*'fV??S Tb'l^'^dia^ illustrating the relationships among a user at a workstation (dient). a todc file 
of the present invention, and a database table in a shared (network) environment * i^ s„w..rti«, 

50 Fig 38 is a blode diagram illustrating an exemplary layout of the lode file of Fig. 3A. the lode file induding 

a plurality of "logical" lock files. 

Fia 4 is a block diagram aiustrating an exemplary layout of a logical lock file of Fig. 3B. 

Ftas. 5A-D are block diagrams Illustrating re-allocation of a logical lode file of the present invention 

Figs. 6A-B are blode diagrams illustrating a method of the present Invention for predidwe reading of files 

55 ^""^^f^^^-^^ g flowchart illustrating a method of the present invention for lodeing shared objects, indud- 
ing the creation and maintenance of a lock file of the present invention. 
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GLOSSARY 



i {disk access): To obtain entry to, or to locate, read Into memory, and make ready for some operation. 
Access is used with regard to disks, files, records, and network entry procedures. 
5 allocate: To reserve memory for use by a program. Programs often need certain system resources such as 
memory or disk space, and they request them as needed from the operating system, 
append: To attach to the end of; this is most often used in reference Id writing to a fie (adding data to the end 
of the fSe). 

block (storage block): A group of similar things- usually bytes of storage or data In disk storage, a block is a 
10 collection of consecutive bytes of data that are read from or written to the d'mk as a group. 

cluster: In data storage, a disk-storage unit consisting of a fixed number of sectors (storage segments on a 
disk) that the operating system uses to read or write Information: typically, a cluster cdnsists of two to eight 
sectors, each of which holds a certain number of bytes (characters). 

directory (and subdirectory): Av^ay of oiganizing and grouping the fSes on a disk; typically, presented to the 
IS user as a catalog for filenames and other dbBctories stored on a disk. What the user views as a directory is 
supported in the operating system by tables of data, stored on the disk, that contain characteristics associated 
with each fSe, as well as the location of the file on the disk. 

entry: A unit of informaUon treated as a whole by a program; also refers to the process of inputting information, 
often in a predetermined form or format, for a computer program to act upon. 

20 field: A member of a row that holds a data value associated with an attribute. 

file: Af De is a conglomeration of instructions, numbers, words, or images stored as a coherent unit whfch may 
be operated upon as a unit (e.g., for retrieving, changing, deleting, saving and the like). A disk file is a basic 
unit of storage that enables a computer to distinguish one set of information from another typically includes 
at least one complete collection of information, such as a program, a set of data used by a program, or the 

25 like. 

f He handle: Awoken- (number) that the system uses in referring to an open ffle. Af ile handle, like a "CB han- 
dle," is a unique klentif ier. 

file name: Af//e name Is a name assigned for identifying a file. 

header Typically the f Iret data in a file, a header stores identity, status, and other data of a file. 
so Index: A stored collection of keys (see below) whteh fociiitate record operations, including searching, inserting. 
- and deleting. Such data structures can include hash tables, binary b-ees, and B-trees. 

input/output: Often abbreviated I/O, input/output refers to the complementary tasks of gathering data for the 

microprocessor to work with and making the results available to the user through a device such as the display 

disk drive, or printer. 

35 key: A data quantity composed of one or more fields from a record. Keys are stored in an Index, and each key 
usually has an attached data pointer that leads to the associated data record. 

location (storage location): The position at which a particular item can be found. A storage location can be an 
addressed (uniquely numbered) location in memory or it can be a uniquely identified location (sector) on disk, 
read (disk read): Read is the operation of receiving input into the computer from a peripheral device, such as 
40 a disk. A read is an I/O operation: data is being output from the peripheral device and input into the computer, 
referencing: Addressing or otherwise targeting a desired object (e.g., file) at a particular (addressable) loca^ 
tion. 

resource: Any part of a computer system or network, such as a disk drive, printer, or memory, that can be 
allotted to a program or a process while it is running. 
45 row: Physically, a row is usually a record In a data file. Logically, a row Is one horizontal member of a table- a 
collection of fields. 

seek (disk seek): The operation of moving the head in a disk drive to a desired site, typfcally for a read or write 
operation. 

size: Typically measured in bytes (i.e. , S-bit units), this value reflects the storage required for an object (in mem- 
50 ory. on disk, or the like). 

storage device: Any apparatus for recording information in permanent or semipermanent form. Most com- 
monly refers to a disk drive. 

table: Usually, a collection of rows all stored in one logical file. 

write (disk write): To fransfer Information either to a storage device, such as a disk, or other output device. A 
55 disk write transfers information from memory to storage on disk. 

The foOowing description will focus on embodiment of the present invention in a multi-user database en- 
vironment The present invention is. however, not limited to any particular exemplary embodiment Instead, 
the teachings of the present invention may be advantageously be applied to a variety of architectures, Appli^ 
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cation of the present invention Is particularly advantageous in those architectures having shared resoufoes. 
^TJ^ng not only multi-user platforms but also multitasking ones as well. In the former, there ,s contenton 
Sr^Sr^s anJ,ng multiple users; in the latter, contention exists among ^^^J^^^'^^'^^'^^Z^ 
Therefore, the following preferred embodiment and certain alternatives are offered for purposes of Ulustrabon 
and not limitation. 
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The present invention may be embodied on a computer system such as the system 100 J- 
includes a central processor 101. a main memory 102 (e.g.. random-access memory or RAM), an input/output 
SSCTer 103. a k^board 104. a pointing device 105 (e.g.. mouse, track ball, pen device, or the hke). a display 
7^ To6. a;;d a non-volatileor Smssstorage 107 (e.g.. hard orfbced disk. optical disk. "?9;«^°P;«^f^ 
orfiash memory). ProcessorlOlindudes or Is coupled toacachememory109 for sti^ngfreque^^^ 

"fbrmatton- memory 109 may be an on-chip cache or external cache (as shown). System 100 may also be 
pS^ vHih SZnal InpuUoutput devices, such as a printing device 108. as desired. The various compo- 
nents of the system 100 communicate through a system bus 110 or similar architecture, as shown. 

Illustrated in Fig. 1 B. a computer software system 150 is provided for programming '^^^^^^^^ '^ 
computer system 100. Software system 150. which is stored in system memory 102 and on disk memory 107 
includes a kernel or operating system 151 and a DBMS 154. OS 151 is the executive or supervisor for the 
20 svstem 100. directing both task management and data management . 

DBMS 1 54. on the other hand, is asoftware subsystem fbr storing, retrieving, and manipulating Uiformation 
in debase tebles (e.g.. tebles 161. 162. 163). Under the command of DBMS 154. the system 100 rec^" 
user commands ani data through user Interface 152. Interface 152 includes a built-in query ^^^^^ 
tor accessina and processing datebase informatton. Additional application programs, such as DBMS appli(»- 
?S.ra^ -leaded- (i.e.. transferred from storage 107 Into memory 102) for execuUon by the 
system 100, particularly for further controlling the operatton of DBMS 154. 

In a preferred embodiment, the system 100 is an IBM-compatibte personal computer ^tem. avalable 
from a variety of vendors (including IBM of Armonk. NY), and operating system 151 is MS-DOS operating sys- 
tem software, avanabie from Microsoft of Redmond. WA. DBMS 154 is preferably a relational datebai» man- 
agement system (RDBMS). More preferably. DMBS 154 Includes Paradox® Datebase Management Sy^m 
(available from Borland International of Scotte Valley. CA). As Interfece 152 Paradox P^'''^**";^ 
or -canvas- and a command menu; a QBE query worksurface Is also provided. Application software 1». n 
turn, include datebase command-language appllcatfons (e.g.. PAL« scripte). which may be executed or other- 
wiseacted upon by the DBMS 154. 

At the outeet. it is helpfiil to understand general techniques for storing information in DBMS 154. In a re- 
lational datebase management system, information is organized into tebles. such as tebto 170 <rf F^. 1C. As 
conceptually shown, teble 170 typically includes horizontel rows or records (tuples) 173 and vertteal coUimns 
or fields 175. Adatebase record includes information which is most conveniently represented as a single unit 
A record for an employee, fbr example, may include Informatton about the employee's ID Number. Last Name 
and First Initial. Positton. Date Hired. Social Secuifty Number, and Salary. Thus, a typical record includes sev- 
eral categories of Informatton about an individual person, place, or thing. Each of these categories, in torn, 
represente a datebase fieW. In the foregoing employee teble. for example. Positton is one field. Date Hired is 
another and so on. With this fbrmat tebles are easy for users to understand and use. Moreover, the f lexibHity 

of tebles permite a user to define relationships between various items of date, as needed. 

45 By employing one or more datebase indexes, the records of a table can be organized in many different 
ways, depending on a particular user's needs. As shown by index 180 of Fig. 1C. for example, an index may 
be constructed as a single disk file which is referred to internally by the system for locating and displayii^ 
records in a datebase file. Index 180 stores two types of Information: index key values 183 and unique record 
numbers 185. An Index key is a date quantity composed ofone or more fields from a record; keys are used to 
50 arrange (logically) the datebase file records by some desired order (index expresston). Record numbere. on 
the other hand, are unique pointers to the actual storage location of each record in the datebase file. In this 
manner, an index for a datebase file is similar to the index of a book, which iiste subject keys and page numbers 
that point to where the aclual information is located in the book. Specifically, an index organizes (logically not 
physically) the records in a datebase file according to the values in one or more fields of Interest As such, an 
55 index may greatly speed up searches (queries) for information. 
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Network Architecture 

While the present invention is operative within a single (standalone) computer (e.g., system 100 of Fig. 
1 A), the present Invention is preferably embodied in a multi-user computer system, such as the dient/server 

5 system 150 of Fig. 1D. Specifically, system 150 Includes a first computer or fSe server 180 and one or more 
second computers or clients 160. In an exemplary embodiment, the clients or workstations 160 are connected 
to server 180 through a computer network 170, which may be a conventksn local area network (LAN). Network 
170 Includes cabling 175 for connecting the server and each workstation to the network. The workstations 
themselves will be simBar to or the same as system 100; additionally, each typically includes an adapter 165 

10 for receiving the network catHe 175. Server 180 may also be sinnllar to or the same as system 100. Because 
the server manages multiple resources for the clients, it should preferably includes a relatively fester processor, 
larger mass storage, and more system memory than is found on each workstation. 

Overall operation of the system 1 50 is directed by a networking operating system 1 81 , which may be stored 
in the server's system memory; in a preferred embodiment, OS 181 includes NetWare®, available from Novell 

15 of Provo, UT. In response to requests from the clients 1 60, the server 1 81 provides various network resources 
and services. For Instance, multiple users (e.g., workstations A, B, and C) may view a database table stored 
in file server storage 183. while another user (e.g., workstation E) sends a document to a network printer (not 
shown). Of particular interest to the present invention is use of system 1 50 for multiuser database access, which 
is described next 

20 

IWuitkuser Database Oiperatlon 

To an end user, using the DBMS of the present embodiment in a networking environment is much like using 
it as a standalone program on a single computer (e.g., system 100). On a network, however, resources may 
25 be shared with other users, with two or nrKwe users often working with the same resource simultaneously; not 
unexpectedly, a given network's rules for file-sharing (I.e., trustee assignments of directories and files) come 
into play. For instance, a user cannot change a table if he or she does not have sufficient network rights to 
the directory the table resides in. Despite these restrictions, network operations renr^in, for the most part, 
transparent to an end user. 

^30 According to the present embodiment, database objects (e.g., tables, forms, reports, and the like) are 

locked by system 1 50 when necessary to ensure data integrity and consistency. Locks temporarily restrict other 
users from accessing an object while the user (lock holder) is using it Typically, these sharable objects will be 
stored in at least one shared directory (e.g., on storage 183). 

The system of the present embodiment provides for both automatic and explicit placement of locks. For 

35 example, a CoEdit mode is provided to let two or more users edit a table simultaneously: each record is auto- 
matically locked when a user begins to edit It and unlocked when the user leaves the record. Alternatively, each 
user can use explicit locks (as described in further detail hereinbelow), thus allowing one to maintain cornplete 
control over the access of others to tables he or she is sharing. 

40 A. Lock Types 

As shown in Fig. 2A, the system of the present embodiment includes a plurality of lock types. Specific 
lock types provided include: directory lock, frjil lock, write lock, prevent full lock, and prevent write lock, in this 
manner, maximum concurrent access is provided and. at the the same time, corruption and loss of data is 
45 avoided. Each lock type will now be described in turn. The reader should note, however, that the terminology 
employed for locks in the system of the present embodiment differe in some regards from that traditionally em- 
ployed. 

1. Directory lock 

50 

With a directory lock (Dir Lock) In place, all users, including the user who placed the Dir Lock, have read- 
only access to that directory's fBes. The user-observable effect is similar to placing a write lock on each table 
in that directory. While the ability to view tables or other objects is not limited, other locks are preferably 
blocked. In this manner, a Dir LocM allows a user to guarantee the objects of a particular directory. 
55 A Dir Lock will typically be placed explicitiy, with the user who places it having read/write/create access 

to the corresponding directory. Explicitiy-placed Dir Locks should preferably not be removed by the system 
when the user exits; instead, they should be explicitiy removed. Again, only a user with sufficient network rights 
(e.g., read/write/create access) to a dir-locked directory can remove the Dir Lock. 
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A Dir Lock may potenttally improve system performance when it accesses objects '"that directory Sp^ 
clf ically the directory alKww the system to treat any object within that directory as a read-only object Since 
Z^ettL a lodced'iirectory can be modified, the system does not have to <*eck o^cte ^^r"'^'^^' 
Thus, the system can use disk-caching to improve perfbrmance when accessing objects in the dlr.*)cked di- 
rectory. 
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2. Full lock 



Accoidlngtothe present embodiment, afulllock (exclusive access) is provWedasthe most restoctive lock 
that the userorthe system can place ori an object once the user starts an operation that requires thesys^^ 

to apply a full lock to an object, other users cannot access that object for any reason including viewing, until 
the lo(* is released. If the object Is a table, a liill lock is preferably placed on that table « «"!''»/T''y- . 

Suppose, for example, that a user is restructuring a Sfocfcteble. As soon as the user begins the operatiort. 
the system places a full lock on Stock and all of Ite associated forms, reporte. indexes, and other family menv 
bere. The todk remains in effect until the restructuring is complete. Afull lock is desired because if someone 
else were to gain access to Stock while the user was restructuring It. Internal inconsistencies could result For 
instance, if another user were viewing the table when the first user detetad a field from 
structure of Stock wouW no longer be consistent with the image of the table being viewed by the other user. 
By emptoying a full took, this problem is prevented. 

3.Wrttelock 

In contrast to aftjil tock. a write lock (shared access) only prevente other users from changing the contents 
of a family of objecte. It does not. however, limit user access to the objecte In the family, for e«'"P'«'|«;;'«*'.[« 
a database table. A write tock also does not prevent another user ttom placing a write tock on the o^^^Cr*"** 
prevents the earlier user from writing). In this manner, a write tock allows other users to access an object at 
the same time the user is doing so. but prevente them from changing that object in any way. 

Suppose, fbr example, a user is copying an Orders table. With a write lock in place. <'l!;^^^'^'f''^^ 
currency view the table but cannot change the table's structure or contente unU the lock is lif ted^uch as when 
the corresponding operatton (e.g.. restructuring) has finished. If the user tries to start an operation for whteh 
a write lock is required, and the object already has a full lock or a prevent write lock on it. or any of the records 
of the object has a record lock, the user cannot continue. The system applies a wnte lock to a table when a 
user who has invoked a user command (e.g.. ToolslNetlChangeslRestert) to perform a query or processes a 
foport based on that table. This lock ensures that the query or report is accurate for the penod of query or 
report execution (because no other users can make changes whle the write lock Is in place). 

Like the full tock. the write lock limite the operattons other users can perform on the teWe. Both are ap- 
propriate when a table must remain stable for a given period. In the instance where the user Intends to perform 
an operation which cannot tolerate changes to the contente of a table by other usere. the user should wite 
lock it If on the other hand, the user intends to perform an operatton that creates, deletes, sorte. or makes 
other substantial changes to a table, he or she should probably place a full lock on It. Any number of users can 
place a write lock on the same tebte. If more than one write lock has been placed on the table, no "sere can 
change the table. If the user wante to guarantee that he or she will have exclusive write access to a table, then 
the user should place a write lock and a prevent write lock (described below) on the table. 

Both full locks and write locks can improve performance. For example, if the user wnte locks a shared 
table many operations are faster because the system does not need to check whether other usere have modi- 
fied it If the user places a fiill lock on a shared table, operations are even faster because the system can buffer 
in memory changes made to the table (rather than having to write each one out as it is made). Thus, write 
tocks and full locks are beneficial even if the user does not expect othere to access the table. 

50 4. Prevent write lock 

Although it does not actually lock an object, a preventwrite lock prevente otheruserafrom locking an object, 
i e from placing a write lock or a full lock on an object (explicit tocking) or performing an operation that requires 
pigment of either a full lock or a write lock (automatic locking). The lock is perhaps most useful in situations 
55 in which modif icatton of a table by two or more usere at once is either required or allowed; for example, when 
multiple useis must be able to modify a table simultaneously (CoEdit mode), if any one user places a write 
lock on a shared table, others cannot make changes to it. If an object the user needs already has either a full 
tock or a write lock on it, he or she cannot start an operation for which a prevent write lock is required. By se- 
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curing a prevent write lock, the user guarantees his or her abOity to access and make modifications to a table 
(with the exception that other users can lock individual records, however.) 

A prevent write lock on a table prevents the system from being able to apply a write lock to that table to 
perform a query or process a report When it encounters a prevent write lock in this situation, the system at- 
5 tempts to process the query or report anyway. If the system detects other users making changes to the table, 
it restarts the query or report; the user attempting to use that table for a query or a report receives a message 
each time the system restarts. A command key (e.g., pressing Ctrh-Break) is provided so that the restarts may 
be aborted. 



io 5. Prevent full loclc 



A prevent full lock prevents other usere from placing a full lock on a table, either automatically or explicitiy. 
Since prevent full locks preclude other users only from obtaining exdush^e access to a table, the user guar- 
antees himself or herself at least read-only access to the table. 

IS A prevent full lock does not prevent other usere from placing write locks, however. When a user performs 
an operation that sets a prevent fiill lock, other usere can start any other operatk>n using the same object that 
does not require a full lock. For example, when a user views a table, the system places a prevent lull lock on 
it This allows other usere to query the table, print reports belonging to the table or invoh/Ing the table, enter 
data into the table, and do any other operations that do not require exclusive use of the table. Thus, a prevent 

20 fUll lock is the least restrictive type of lock and, therefore, allows the highest level of concurrent access. 



8. Lock Interaction and compatlbllKy 



The system of the present embodiment has virtually no limit to the number of locks that it can place si- 
23 multaneously on an object Only certain locks can coexist, however. The locks that can coexist for a single 
object may be summarized by the following table. 



30 

F&olH Ijock Write lock IPrevenlt IPrevema 

write loelk ftHDl Doclk 

Full lock 

35 

Write lock ✓ ✓ 

Prevent write ✓ ✓ 

lock 



40 



Prevent full c/ ^ 

lock 



45 ^ 

As shown, full locks are incompatible (i.e., cannot co-exist) with all other locks, including record locks. Write 
locks, on the other hand, are compatible with prevent full locks and other write locks; however, a write lock 
does not guarantee that the user will be able to change the write-locked table. Prevent write locks are com- 
50 patible with other prevent write locks and with prevent full locks. Finally, prevent full locks are compatible with 
all locks except full locks. 

Referring now to Figs. 2B-C, exemplary lock interactions will now t>e described for concurrent operations 
in a multi-user environment As shown, multi-user system 200 includes clients 210 connected to server 230 
through network 220. An information table 233, which Includes a plurality of information records, is stored in 
55 a shared drectory of the server 230. 

Clients 210 includes tisers A-D, each of which desires a certain operation to be performed on the table 
233. User A. for example, wishes to perform an exclusive write (private edit) operation on the table 233: this 
operation requires a full lock to be placed on the table 233, as shown in Fig. 2C, Since a full lock is incompatible 
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with other lock types, no other use.3 (i.e.. Usera B-D) can have access to the table 233 during this exclusive 

°'**SS'user B wishes to perform a view (read) operation on the table 233. This requires that a Prevent ftjl 
lock be placed on the table as shown in Fig. 2C. The preventfull look Wocksfull ^^'^^ 
an exd^e write operation (e.g.. by User A). A prevent full lock does not however, block other lock types, 
thus, other operattons (such as those of User C and D below) may continue to be Performed concurrently 

User C d^s a copy operation, such as copying the source table 233 into a target table (not shown) Jcj 
the operatton to be successful, the contents of the source table must not change during the operation. With 
a write lock in place, as shown in Fig. 2C. this requirement is met. Specifically, other usere are given read- 
oiSTaccess to the table and. thus, cannot change the contents of the table. As the lock » incompatible wrth 
both full locks and other write locks, the lock would block the operations of User A (wnte operation) and User 
D (CoEdit operation, described next). 

user D wishes to perfbrm a -CoEdiT operation. I.e.. editing or writing to the table 233 but only one teco^ 
at a time As shown, a prevent write lock is placed, thus preventing other users from starting operattons which 
require either lull or write kicks. Hence, the lock wouM block the operattons of User A and User C. 

C. Automatic and explicit locks 

Locks can be placed either explfcWy by the user or automatlcally by the system. The former is accom- 
plished by issuing one of a number of lock commands: the latter Is im«)ked in response to a parttoularoperatton. 
Each of these wni now be described in further detail. 
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1. Automatic locks 

In a multi-user environment, the system of the present embodiment automatteally locks objects during ev- 
ery operation where there could be contention for a resource. In general, tocks that am ^^^^^^^ 
by the system are the weakest (least restrictive) possible, yet sufficient for maintaining data integnty for the 

duration of the operation. * a * w^^^^x 

This is best explained by an example. When a COPYoperation is invoked (e.g.. copy a famly of objects), 
the system places a write lock on the source table and on each of the obiecis in its family. The wite lock pre- 
vents other users from modifying any member of the source family during the copy operation, but does not 
prevent them from accessing family members for readonly operattons such as viewing. The system a^so Places 
afuli lock on the destinatton table imwived In the copy, this prevents other users from accessing the <lesft"a^o" 
table in any way during the copy, in this instance, the destination table often does not even exist at the start 
35 oftheCOPYoperalion. Nevertheless, the system can lock it 

The ability to took a nonexistent resource is very useful. For example, this allows a user to prevent others 
from creating an object during the period of time the user is doing so. Also, other users are prevented from 
deleting a table out from under the user (i.e.. deleting the table between the time the user creates it and the 

^^lheVs!emon^6^»sent embodiment provides four types of automatte locks: femily locks, record locks, 
group tocks, and write record tocks. With the exception of record locks, only the system (not the user) can place 
these locks and it does so only In specific circumstances. Since the locks are automatically placed on a user s 
behalf they do not restrict what he or she can do with an object instead, automatic locks only restnct what 
others' can do. However, if others are performing operattons that lock an object those locks restnct what the 
user can do (and hence what automatte locks may be placed). Each automatic lock wHi now be described in 
turn. 

a) Family locks 

Afamily lock is a write lock placed on all famffly members of a table, including the table itself. Afamily lodt 
altows the same level of concurrency as a write took: it pravente other users from making changes to any of 
the family objects. 
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b) Record locks 



CoEdit mode allows multiple users to edit tebtes simultaneously and mteractively. thus a lock is needed 
for tocking individual records of a table. When the user coedits a record, the system automatically tocks it. When 
the user finishes making changes to the record and moves the cursor to another record, the system automat- 
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ically removes the lock and posts the user's changes to the table. A LockKey connmand is also provided for 
explicitly locking or unlocking a current record. User feedback is provided for indicating when a record is locked. 

When a record is locked, other users can view it but cannot modify or delete it If the user tries to change 
a record that is locked by another user, the system informs the user the name of the user who has locked it 
The user also cannot place a write lock on a table that has locked records; thus, the presence of at least one 
record lock in a table has the same effect as a prevent write lock on the table. 

c) Group locks and vvrito record locks 

Group locks and write record locks work in tandem to maintain referential integrity between tables that 
are linked by key fields through a multi-table form. The system automatk:ally applies both kinds of locks when 
auser coedits using a multi-table form that expresses a one-to-many or many-to-many relationship, tike record 
locks, group and write record locks come into play only in a coediting environment and occur at the record level, 
not the table level; this approach providesthe greatest amount of concurrent use. These locks are in fact record 
locks, locking both linked master and detaS records. 

If the user begins to ooedit the primary key of a master record in a multi-table form, the system places a 
group lock on the detail records owned by that master record. This prevents other users from altering the detail 
records while the user alters the master (even if they are coediting the detail tables in table view, a single- 
table form view, or another multi-table form instead of with the same multi-table form one is using). Once the 
user finishes editing the master record, he or she should move off the master record - to another master record 
or to a detail record, or press the LodcKey command key (e.g., Alt-L), the system automatically makes the same 
changes to the linked fields of the detail records. 

Write record locks work from the opposite standpoint The system applies a write record lock on the as- 
sociated master record whenever the user opens a record or coedits an existing detail record in a multi-table 
form. The write record lock thus guarantees that the associated master record is not deleted or changed by 
others while the user is coediting the detaB group of records (even if the other users are coediting the master 
table in tat>le view, a single-table form view, or another multi-table form instead of with the same multi-table 
form the user is currently using). 

The system of the present embodiment locks objects intelligently. In every situation, the least restrictive 
fock consistent with the operatton the user is performing is applied. The system therefore allows the greatest 
possible access to tables, forms, and reports by all users, in addition, enhanced concurrency for processing 
queries and reports are provided. For example. In the case of an INSERT query, when the user places the 
Query form on the desktop, the system places a prevent full lock on the table the user is querying. When the 
operation is conf rmed by the user, the system of the present embodiment places a fiill lock on the table while 
It processes the query. In some operations, the system may place locks on two or more tables simultaneously. 
For example, when the user copies a table (i.e., from a source table to a target table), the system automatically 
places locks on both the source and the target table. 

2. E^liclft locks 

Even though the system of the present emtxxiiment provides automatic locking for every operation, in rrnDst 
multi-user applicatkins. the user will want to use explicit locking convnands to control access to resources in 
addition to depending on the automatic locks. For example, the user might want to lock a table explicitiy when 
he or she wants to make sure it is available (e.g.. for continual updating). The explidt lock commands give the 
user more control than the automatic locks and also make it easier for the user to handle situations where a 
user cannot place a lock because of contention for a resource with other users. 

In a preferred embodiment the user can explicitiy lock and unlock tables in two ways: by menu commands 
(e.g,. using ToolslNetltock and ToolslNetrPreventtock) or by using database application programming com- 
mands (e.g.. using LOCK and UNLOCK commands). In this manner, the user can place locks of varying 
strengths on objects, including (in order of increasing concurrency): full lock, write lock, prevent write lock, pre- 
vent full lock, and directory lock. 

Both explicit and automatic locks can be active at the same time on the same object For example, if the 
user employs the LOCK command to explidtiy place a full lock on the Orders table and then uses the COPY 
command to copy Orders to Newords, he or she places both an explicit full lock and an automatic write lock 
on Orders. From the perspective of other users. Orders appears to have only a foil lock on it, since that is the 
stronger of the two locks the user has placed. At the end of the COPV operation, the automatic write lock dis- 
appears, leaving the user's explicit foil lock intact 

Other users can also place locks, both explicit and automatic, on objects that the user has locked. How- 
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ever, locks so placed must coexist with existing locks (see lock compatibility discussed atowe). In a preferred 
embodiment, attempts to place object and record locks, both automatic and explicit, are honored on a f Irst- 

oome. first-served basis. . . , ^. . . j 

Whether placed by the system automafically or by the user expiteHy, a lock of a certain kind retains its 
5 same effect. For example, suppose the user want to enter new data into a Customer table (e.g., using a Mod- 
IfylDataEntry command), the system places a prevent full lock on Customer. This prevents other userefrom 
changing the table's structure or from editing the table while the user is entering data into it. Then, when the 
user posts the new records to Customer, the system of the present embodiment places a prevent write lock 

on the table. ^ . . , .u * 

10 But suppose another user places a vwite lock on Customer while the first user is entenng data. In that case, 
when the user posts, he or she does not get suff toient control over the table to finish his or her data entry op- 
eratton If the user anticipates such a situation, he or she can explksWy place a prevent write lock on the Cus- 
tomer table before the user begin data entry. This guarantees that the table will be available to the user when 
he or she wants to complete the data entry operation. 

Explidtlocks are used mostoften for multi-user applicattons, when the application developer needs precise 
control over access to tables, forms, and reports, in practice, the user should use explicit locks sparingly, since 
they might needlessly prevent other users from accessing objects, instead, the user should emptoy the auto- 
matic locks of the system to maximize concurrent access. 

20 internal Operatton of Locks; Lock Fil— 

The system of the present embodiment manages locks by creating a special took file for each shared di- 
rectory that users access. As shown in Fig. 3A. for example, a client 310 desires access to a shared table 333 
which resWes on a f Be server 330. The system 150. in turn, creates a "physical- lock file 350 for storing vanous 

25 tocking and related concurrency informatton. in a preferred embodiment, the file 350 exists as a separate disk 
(i.e.. -physical") file which is stored on the server 330. preferably In the same network directory as the table. 
Each file 350 stores at least one logical tock file 380 having tocking information specif to to the shared table 
333 (and its family membere). Unlike the physical files, however, the logical lock files do not exist as separate 
disk files. At the conclusion of a multi-user session for a shared directory, the corresponding physical file, to- 

30 gether with its logteal files, is deleted. 

A. Physical lock file layout 

Referring now to Rg. 3B. the structure of the physical lock f Be 350 will be described in further detail. As 
shown, physical lock file 350 stores one or more logical tock f itos 380. For managing these files, physical lock 
file 350 also includes a header 360 and a directory 370. Each of these will be described In further detail. 

Header 360 includes housekeeping information. For example, header 360 stores a "highwater mark" value 
365 pointing to the iocatten (end of the physical lock file) where new logical lock files or resized existing logteal 
files are to be allocated (appended). The highwater location varies dynamically according to the number and 
size of logical lock files present in the f Be 350. Spectf kaily. as logical tock files are added to the physteal f Be. 
the highwater mark is adjusted upward. Since togtoal lock fUes are preferably never physically deleted, how- 
ever, the mark will not be adjusted downward. 

Header 360 also stores a Dir Ljock 367. whfeh is a flag Indicating whether a directory tock Is in place. Ac- 
cording to the present embodiment, a directory lock provides read-only access to all objects present in that 
45 directory (as described hereinabove). When present the directory lock is controBing: specif teally. other locks 
(e.g.. record locks, table locks, and the like) are simply ignored (or btocked). With a directory lock In place, the 
remaining Informatton stored in the logical lock files Is Irrelevant and. thus, need not be read. 

Access into the logical lock files themselves is provided by dire<*)ry 370. The directory stores an entry 
(e.g.. entry 371 ) for each logical lock f He. Each entry includes a name, such as a siring or handle corresponding 
so to a family name, and a pointer to the particular logical lock file being referenced. In this fashton, the directory 
serves as an index into the avaMable logical lock files for a given physical lock file. 

When a workstation is first accessing an object of a family (e.g., table), it checks the directory 370 for the 
existence of a corresponding logical lock f He. in the instance that a logteal tock fBe does not exist (no directory 
entry is found), one is created on the fly. Specifically, a new logical lock file is appended to the end of the 
S5 existing logical lock files (i.e., at current highwater mark 391) and an entry is made In the directory 370 for 
referencing the file. To speed up subsequent accesses to that logteal lock fOe. however, a workstation need 
only memorize (e.g., store in local memory 102) the address or offset of that logical fOe. The system in fact 
remembers (i.e., stores in memory 102) the respective location of each logical tock file read. In this manner, 
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the directory need only be read once for each particular logical lock file accessed. 

The logical lock files themselves (e.g., file 381, file 383, file 389) stores concurrency Informatton associ- 
ated with a particular database famfly. Each is allocated at an inital tAock size. Depending on operating system 
employed, blocks may advantageously be allocated as a multiple of a common disk unit (e.g., as byte dusters 

5 of 512, 1024. 2048. and the like). Normalizing the logical file to pre-selected block sizes not only simplifies 
directory maintenance but also speeds access to the concurrency information contained within the files. The 
latter advantage is now examined in further detail. 

To minimize allocation of additional storage for a logical lock file, each block typically includes a free or 
buffer area, at least initially, in which the number of logical file entries stored can grow and shrink (i.e., without 

10 continually allocating additional blocks). When the initial storage capacity of a logical file (e.g., IK) has been 
exceeded, the file is effectively moved to the physical took file 350 with a larger block size (e.g., 2K). After 
examining the detailed structure of logical lock files, the growth of physical lockf Oes will be descrit)ed in further 
detail. 

IS B. Logical lock file layout 

As shown in Fig. 4, the logical lock f fle or table Itself includes a plurality of entries for specifying the con- 
currency information of associated famOy members. A preferred layout for a logical lock file 400 of the present 
embodiment includes a header 410 and lock data 420. Basically, the header maintains housekeeping infor- 

20 mation. Lock data 420, on the other hand, stores a variable number of registratton entries; exclusive access 
to these data may be achieved by locking a selected byte ("byte locking") of the lexical file. 

Header 410, as illustrated, includes Famfly or DIr ID 411 and NBIock413 fields. Dfar ID 411 stores informa- 
tion Identifying thefamDy of database objects which Is associated with the logical file. Family members include 
a database table and its related objects, such as forms, data validation, reports, indexes, and the like. In a 

25 preferred embodiment, the base name of the database table identifies all related members of a given family. 
A customer table (e.g., cusiomer.db) may have associated with it a customer form (e.g., customer.^ and a data 
validation file (e.g., customer.val). Any database object which shares the same base neune €is the table (e.g., 
all database objects in the family Customer.<^) also employs the same logical lock fBe. While the base name 

- may be stored as the Dtr ID (e.g., as a text string), a simple handle scheme is preferably employed instead. 

30 All told, the approach faclitates the locking of associated members, such as locking forms and reports. 

In addition to storing family information, the header 410 preferably includes information capable of redir- 
ecting a workstation to another logical file. As shown, header 410 includes the NBIock field 413 for storing the 
number of storage ft)locks allocated for the logical lock file. In the event that a logical lock file is "moved" to a 
new storage block(8), NBIock 413 is set equal to "DELPENDING" (i.e., delete pending), which in a preferred 

35 embodiment is defined by the value QxFFFD. In this Instance, a pointer referencing the new logical file block 
is also stored (e.g., by overwriting the data no longer needed). A detailed example employing this mechanism 
is set forth below In Figs. 5A-D. 

Logical lock file 400 stores the actual concurrency information of interest as registration entries (e.g., re- 
cord 430) in the data area 420, which includes a single entry header 421. Each entry (entry or record 430) 

40 includes information completely describing a single lock or concurrent action. The single header 421 stores 
status or historic information for the entries, including size 423. soft (special critical section) k>ck 425, version 
427, and the like. 

Data 431, on the other hand, includes Type ID 433, Length 435, User ID 437, and Registration Data 439 
fields. Type ID field 433 includes information (e.g., a special ID code) identifying the general type of information 
45 maintained. For example, an entry may be identified as a record lock, group lock. Image area, table lock, or 
the like. User ID 437, on the other hand, identifies the actual holder (user) of the entry. Preferably, this infor- 
mation is stored succinctiy as a handle (integer) referencing a network user. 

Length field 433 stores the length of the record or entry. While most records may be standardized to a 
fixed length, it is also desirable to store variable information, such as a key value (e.g., for group locks). The 
50 length field provides this capability. Moreover, the length information fosters compatibOity: a system need not 
know the length of different entry types beforehand. Instead, the information is simply read as needed. 

Registration Data 439 stores the specific lock information for the entry. In a table lock, for example. Data . 
439 stores one of a full lock, a write lock, a prevent full lock, or a prevent write lock. In a record lock, on the 
other hand, Data 439 stores the record number of the record being locked. To lock a set or range of records 
55 (a group lock). Data 439 stores a key value (e.g., $3,000). This is useful for a one-to-many relationship: the 
detail table may be convenientiy locked by the key of the master table. 

Since different users may have disparate views of the same non-keyed tables, Data area 439 stores in- 
formation for synchronizing non-keyed tables. If one user, for example, inserts several records before {above) 
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a current record for a second user, the record number for that current record, as seen by the second user, is 
no longer valid. According to the present embodiment, in this instance the workstation modifying the shared 
table examines the lock file to update the image area for other users. In a preferred embodiment, a skew or 
synchronization factor is added to the Image area entries of others, thus adjusting the relative position of the 

5 records as viewed by different users. For keyed tables, in contrast, sufficient informatton already exists in the 
primary key (i.e.. index file) for adjusting relative positions, all without accessing lock files. 

In actual operation, the individual workstations effect locks by adding and deleting relevant registration 
entries to the corresponding logical lock file. To add a full lock to a Customer table, for example, a workstation 
would register with the Customer logical lock f He an entry specifying a table lock (Type ID), the holder (User 

10 ID), and a full lock (Lock Data). Before such as entry is permitted, however, the system verifies that no incom- 
patabilities occur with existing entries. In this manner, a logical lock file serves as a list which may be sequen- 
tially processed for determining lock Informatton for various objects of a family. 

As locking information for a family of objects may be continually changing, logical file 440 includes a buffer 
zone or area 440 for accommodating a variable number of registration entries. In this fashion, related concur- 

15 rency Informatton may be maintained together for rapid sequential access. According to the present embodi- 
ment however, additional methodologies are provkled for even further improving access. 

C. Logical lock file growth 

20 The tocking scheme of the present embodiment Includes novel techniques for controlling the addition and 
revision of locking Information. To optimize performance, a parttoular logical file Is preferably always accessed 
in one contiguous read, thus minimizing the perfbrmance penalty for additional I/O operations. This may be 
achieved by maintaining logical lock files as self-contained blocks. 

According to the present embodiment, a logical lock file is created at an initial block size, such as one kilo- 

25 byte (1 K). The entries of the file are stored compacted, without any particular linking information. For example, 
new entries are simply appended to the end (Le., at the beginning of the free area); dead entries, on the other 
hand, are squeezed out during compaction of the logical file. During this time of actually modifying (writing to) 
a logical file, the accessing workstation maintains exclusive control over that file (e.g., through byte locking). 
Growth of logical files (and hence underlying physical file) is controlled as follows. When increased storage 

30 b required (e.g., when numerous concurrsnt users are present), the storage currentiy available for the logical 
lock file (e.g., 1 K) may be insufficient; additional storage must be allocated. According to the present embodi- 
ment, the storage for the logical lock file is not increased by chaining a new storage block to the old one (e.g., 
through a pointer). Instead, an entirely new logical lock file, one having a larger block size (e.g-, now at 2iq, 
is created in the physical lock file. 

35 Referring now to Figs. 5A-D, this operation is illustrated. Fig. 5A illustrates a physical lock file 500a having 

a plurality of logfcal tock files. Suppose, for example, that the storage requirement for logical tock fie A (501 a) 
exceeds that currentiy allocated for the file. As shown, additional space 503a is available towards the end of 
the physical lock file 500a 

According to the present embodiment, additk>nal space for logical lock file A is allocated as follows. As 

40 shown in Rg. 5B. an entirely new logical lock ffle A (501b) is appended to the end of the existing lock fBes 
(i.e.. at the beginning of available space 503a). The new file (501b). which is allocated twice the original size 
(i.e., now with two IK storage blocks), stores current entries as well as any new entries for the logical file. As 
shown, available or free space 503a shrinks accordingly (now space 503b). To complete the operation, logical 
lock file A (501a) is now marked as an invalkl block (501a'). The invalidation is achieved by enabling a pointer 

45 which points from the old block 501 a' to the new block 501 b. Two items of Interest are memorized: a) allocated 
or maximum size of the logical lock file (ends at 504b), and b) actual size used (bytes of content) in the logical 
lock file (ends at 505). The latter Is used for predictive reading, which is fully described below. 

At this point, two different perspectives exist for the physical file. For the current workstation (i.e.. the one 
effecting the change), the physical lock file appears as file 500b of Fig. 5C. Specifically, logical lock file of 

50 interest, file A (501 b). has a new starting location 502b and ending location 507. Thus, access to file A will 
include a seek to location 502b. followed by a readfirom locatton 502b to no more than location 507 (and typ- 
ically much less). 

Other workstetions. specifically those which have not accessed file A since ite re-allocatton. will have an 
outdated perspective. As shown by Fig. 5D, a subsequent workstation will initially access {seek to) starting lo- 
ss cation 502a - where it expects to find File A. Once at that location, however, it finds that the block (now block 
501c) no longer contains valid locking information; instead, the block contains a pointer to new storage block 
(501b) where the newly allocated file A is stored. In essence, the other workstations are told that the logical 
lock file has been moved from the first btock to a larger second block. Once a workstation has received this 
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notification (that it should access file A at the second block), the workstation wfli then continue accessing the 
second block for needed locking information, at least until It is notified of yet another change. Specifically, the 
offset of interest (offset 502b) is stored locally (e.g., in memory 102) so that subsequent access to the f 3e A 
may occur directly, i.e.. without traversing any pointers or re-reading the directory. In this manner, lock files 
5 remain contiguous and, at the same time, file I/O operations are kept to a bare minimum. 

D. Improved access: predictive reads 

By "predlctively reading" the logical lock files, additional performance gains may be realized. Specifically, 
10 each workstation remembers the location and size of a lock file and a predictive amount. On the next read of 
that file, the workstation seeks to the memorized location and reads the memorized size plus an additional or 
predictive amount (size read = size from prior access + predtotive amount). In the event that a prior size is not 
known, one may be simply guessed (e.g., defaults to 80% of block size). 

Suppose, for example, a workstation accesses a logical lock file having a size of 550 bytes. On a subse- 
ts quent read of this file, the workstation may predict that the lock file wifl grow in size - that new registrations 
entries will be added. Instead of just reading 550 bytes, the system may read more, 800 bytes for example, in 
the event that the logical file is shrinking, on the other hand, the system will read less. In this manner, a logical 
lock file win always or substantially always be read in a single I/O operation. 

Referring now to Figs. 6A-B, the operation of predictive reading is Illustrated, in Fig. 6A, logical lock file 
20 600a stores a plurality of entries, spanning from beginning offset 601 to ending offeet 603. Thus, the current 
size (of entries) Is Illustrated by size 607. Upon subsequent access of the file 600a, a workstation could simply 
read Just the information between offsets 601 and 603. If additional records exist beyond offset 603. however, 
that approach would require a second read operation, thus incurring a substantial performance penalty. By 
adding a predictive amount 609 to the previous size 607, the system of the present embodiment intelligentiy 
25 reads the file, thus minimizing or eliminating additional file I/O operations. 

This approach is further Dlustrated in Fig. 6B. To read logical file 600b, the workstation first seeks to lo- 
cation 601 . Next, a single, sequential read is performed from location 601 to location 61 3. In this manner, entries 
added since the last read (e.g., new entries 611) are read without an additional file I/O operation. While more 
bytes will typically be read than is absolutely necessary, these additional bytes may be simply ignored (i.e., 
30 need not be further processed) -the performancepenaltyfortwoormore reads of a physical disk substantially 
' : outweighs any disadvantage from an intelligent single read of additional bytes. For networking systems (e.g., 
Noveirs Netware) where the performance penalty for file I/O operations is high, the methodology is particularly 
advantageous. 

In the unlikely event that a predictive read fails to read sufficient information or faults (e.g., when the file 
35 has moved), an additional file I/O operation may be performed. The worst possible case for predictive reading, 
however, fares no worse than conventional techniques. Moreover, the predictive amount and the logical file 
block size may be dynamically adjusted to minimize or virtually eliminate such faults; at the same time, when 
the size of the lock file decreases, the size of the predictive read decreases as well. 

40 E. Byfte iocEcIng 

Using the techniques of the present embodiment, once a logical lock file has been created, each work- 
station knows where to find the locking information. To actually alter the contents of a logical lock file, however, 
a workstation may obtain exclusive control over the file. A byte lock is employed for this purpose. 
45 Byte locking or record locking occurs as follows. Given a file, a particular byte may be locked by a single 

user. For each logical lock file, a critical section (i.e., a portion that only one process can modify at a time), 
such as a particular byte, may be specified for locking. In operation, a byte lock is held for a very short period 
of time - only the time required to perform the actual update of a logical lock fie. After processing the lock 
information, the system simply releases the byte lock (instead of closing the file). In this manner, the system 
so requires only a single file handle for the physical lock file (instead of one for each logical lock file). 

Since locking of a byte is an exclusive event, a mechanism is provided by operating systems for conti-olling 
conflicts. In Microsoft's MS-DOS, only a simple system is provide: if an attempt is made to lock a byte which 
is already locked by another, the system only returns an error code. For these types of systems, therefore, a 
polling mechanism is employed to resoh^e conflicts of byte locks. In more advanced networking environments, 
55 such as Novell's Netware, a queuing mechanism is typically provided, thus eliminating the need to poll byte 
locks. In these latter systems, the lock requester may wait for any pre-existing byte lock to be removed (e.g.. 
within a specified timeout period). 

All tdd. the system adapts to the growing and shrinking of lock information in the logical f fle. By employing 
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byte locking technique. Individual lockfUes may be managed with a minimum of system overhead. 
F. Preforrad method far locking objects 

Referring now to Rg. 7A, a method 700 for locking o^ecls wiU now be described. In step 701. a reqMest 
is received for access to a shared object of a family (e.g.. table or related member) At step 702. the method 
700 checks to determine if a physical lockf lie exists for the requested object. If no. then one » created ms^ 
m-aherwise (yes at step 702). the method skips step703. At step704. the system checksfort^ 

of the logical ioi f ile: either the address is already known (l.e.. stored locally f^mprev»u8 'e^d^^^nd^ 
the dirertory 370. If no address can be found (i.e.. the file does not exist), then the logjca^f .le » 
allocated) in step 705. In this instance, the highwater mark (of Fig. 3) is set to pomt Id the end of the last file 
block Otherwise (yes at step 704). the method skips steps 705 and 709. 

AfS Seating a new phj^ical and/or logical f Be and updating the high water mark, the n«thod preceeds 
to steo 710 If the logical file is found (yes at step 704). however, then In step 706 the method reads the file 
after seeking to its offset (beginning address). Since it can be predicted that the lock tie will grow, the step 
performs a larger sequential (predictive) read than would normally be done. In particular, the read indudes 
an amount equal to the size from the prevtous read plus an addittonal (predictive) amount In a streamline em- 
S^lment. the predictive amount may be arbltrerily set and then empirically adjusted until an oPt-n^;;^ va^ « 
obteined: specifically, the value is increased or decreased until read l^utts are minimized. In a more oomp^« 
embodiment, expert system techniques, employing a knowledgebase and inference engine, may be provid^ 
fbr intelligently determine an optimum value. THe design and Implementetton of expert systems Is ^own in 
the art See e.g.. Wahr. P. Expert Systems: Techniques, Tools and Applications. Addlson-Wesley. 1986. 

At step 707. If the currently accessed block of the logical look f Ue Is Invalid, then the system will ump to 
the new vaUd logical f He block (by reading a pointer to it) at step 708; the new address is now remember (stored) 
locally. To increase performance, often-accessed information, such as offset address is stored locally (..e.. 
in the system memory of the workstetion). preferably in cache memory (e.g.. memory 1 09). If the current logical 

file is not invalidated however (yes at step 707). then step 708 is skipped. 

At step 710. the locking information within the logical lookfile of interest is processed; th« step is described 
in further deteil below in Fig. 7B. At step 711. the method remembers the current sia» of the logical lockflte^ 
Again, this Infbrmatton is preferably stored in local memory. At step 712. the method loops "ack to step 706 
for new operations with known families. At step 712. the method loops back to step 704 for new op^^ic^ 
unknown families. At step 714. the method loops back to step 701 fbr operations m a new directory. At the 
conduston of these steps, the method has completed. ^ ^ •.• ^ a».»<« 

Referring now to Fig. 7B. the individual substeps of step 710 (from method 700) are illustrated. At step 
751. the method reads registration entries and applies the information (i.e.. effecte the "«f«««^^y^<=''?J^ 
cordlngly. At step 752 if new entries are to be update, then method proceeds to step 753 to tft^^'hether there 
is sufffeient space present in the fie block. Otherwise (no at step 752). the method jumps to ateP 757. ifi^ 
sufficient space existe for new entries, then a new block is allocated at step 754 for the logical lock file. The 
old fae block is im^alidated. and a pointer to the new block is stored. Upon reading the pointer to the new ad- 
dress, subsequent workstations will memorize the new address (see step 708). At step 755. the highwater 

is increased by an amount equal to the newly allocated file block. If. on the other hand, sufficient space d^ 
exist at Step 753. then steps 754 and 755 are skipped. At step 756. the method updates theregistiation entries, 
as needed, by writing to the logical lock fite; a byte lock is temporarily employed on the logical "e "unng the 
step. Finally at step 757, if additional operations are to be performed on this ot^ject family, then the method 
loops to step 751 accordingly. Otherwise, the method retorns. 

While the invention is described in some deteil with specific reference to a single preferred embodiment 
and certain alternatives, there is no intent to limit the invention to that particular embodiment or those specific 
alternatives. For example, application of the present invention is particularly advantageous in those archrtec- 
tures having shared resources, including not only multi-user platforms but also multi-tasking ones as well. Thus. 

the present invention is not limited to any one of the foregoing exemplary embodiments. 



Claims 



In a computer system having a memory and a mass storage means, a method for storing and retoeving 

a plurality of information files, the method being characterised by: 

(a) storing the plurality of information files as a single mass storage file which includes at least one 
storage block, of uniform size, for each of the information files, and each said information frte having 
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a mernorized f He size when it has already been read: and 

(b) retrieving a previously retrieved information file from the mass storage means by 
(i) locating a storage block or blocks having the desired informatton f fle, and 

(il) reading from the storage block or blocks an amount of data equal to the previously memorized 
5 file size of the information file plus a predtetive amount 

2. A method according to daim 1, and comprising: 

(c) processing said retrieved information f fle in the memory, said processing Including adding and/or 
deleting data entries firom the information file: 

(d) storing the processed information file in the mass storage means; and 

(e) storing In the memory a new size for the processed information file. 

3. A method according to daim 2, wherein step (d) indudes the substep: 

locking at least one byte of the least one storage block storing the information f De, whereby exdu- 
sive control over the at least one storage block is obtained. 

4. A method according to daim 2 or 3, and comprising: 

(f) repeating steps (b) with said new size as the file size. 

5. A method according to any one of the preceding daims, wherein step (a) includes the substep: 

^ storing with the single mass storage f De a directory of said Information files, said directory induding 

a plurality of entries each of which stores a locatk>n of an informatk>n file within the single mass storage 
file. 

6. A method according to daim 5, wherein said locating a storage block indudes: 

25 searching the directory for an entry referendng the desired information f ae; and 

when the entry is found, retrieving from the directory entry a starting location for the desired in- 
formation file. 

7. A method according to daim 6, wherein step (b) further indudes: 

30 when the entry is not found, appending a new storage block to the single mass storage file for stor- 

ing a new informatton file. 

8. A method according to daim 6 or 7, wherein step (b) indudes: 

seeking to the starting locatkm of the desired informatk>n file; and 
35 in a single input/output operatton, reading an anrtount of data equal to the file size plus the predtetive 

amount 

9. A method according to any one of the preceding daims, wherein said mass storage means indudes a 
cluster size in which information is retrieved, and wherein each of said storage blocks has a size which 

^ is equal to or a multiple of said duster size. 

10. A method according to claim 2, 3 or 4, or to any one of daims 5 to 9 when appended to daim 2, wherein 
after step (e): 

If the new size exceeds the at least one storage block size, 
<l) invalidating the at least one storage block, 

(il) appending to the single mass storage file a new block having a size larger than that of the invalidated 
storage block, and 

<iii) storing the information file in the new block. 

11. A method according to any one of the preceding daims, wherein said predictive amount is arbitrarily set 
^ to an initial value and then empirically adjusted to an optimum value. 

12. A method according to any one of the preceding daims. wherein said predictive amount is increased or 
decreased for minimizing read faults. 

55 13. A method according to any one the preceding claims, wherein the system has a plurality of resources 
concurrentiy accessed on the mass storage means, the method comprising grouping related ones of the 
resources into individual families, and wherein the indivklual files are logical lockf iles, contained in a phys- 
ical lock file, for storing information for accessing the resources of respective ones of said families. 
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14. A data processing system for storing and retrieving information comprising at least one storage means 
(107, 109) for storing information as separate data f Ues and at least one memory (102) for processing 
Information, the system being characterised by: 

means arranged to store selected ones of the data files together as a single data file in said at 
least one storage means, said single data file Including at least one stonage block for each of the selected 
data flies, the storage blod^s having a uniform size, and each selected data file having a size stored "m 
said at least one menK>ry when it has been retrieved; and 

processor means (101) arranged to retrieve a desired one of the selected data files Into said at 
least one memory by transferring from said at least one storage means (1 07,109) an amount of Information 
greater than the stored size of the desired data file. 

15. A system according to claim 14, wherein said amount of Information transferred Is less than or equal to 
the size of said at least one storage block. 

16. A system according to claim 14 or 15, wherein said processor means Includes means arrange uniquely 
to identify said single data file by a single file handle. 

17. In a computer system having a plurality of resources concurrently accessed on a shared storage means, 
a method for managing access to the resources, the method being characterized by: 

grouping related ones of the resources into Individual families; 

storing in a physical lock file in the storage means a plurality of logical lock files, each logical lock 
file storing information for accessing resources of a single family, said physical lock fie having a plurality 
of uniform storage areas each of which stores no more than one logical lock file; and 

accessing each resource of interest by reading the logical lock file for the family of the resow^e. 

18. A method according to daim 13 or 17, wherein each resource of a family shares a common base name. 

19. A method according to claim 13. 17or 18, wherein said physical lock file Includes a header storing a value 
for specifying a point of allocation for appending additional storage areas to the physteal lock file, where- 
upon the value Is increased for each new storage area appended. 

20. A method according to claim 19, wherein said storage means Includes a plurality of shared directories for 
storing resources, and wherein said header includes a flag for preventing modification to any resources 
stored in a shared directory. 

21. A method according to any one of claims 13 and 17 to 20. wherein said physical lock file includes a di- 
rectory for locating logical lock files, said directory including a plurality of entries each of which stores a 
storting address for one logical lock file. 

22. A method according to any one of claims 1 3 and 1 7 to 21 , wherein each said logical lock file includes a 
header storing an ID for a single famUy and a flag for indicating that the logical lock file has nrK>ved. 

23. A method according to claim 22, wherein said logical lock file header stores a pointer to a new logical lock 
file if saki flag indicates that the logical lock file has moved. 
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EFFECT 



Places a lock on a shared directory, allowing 
read-only access to the directory's files. 

Places a complete lock on objects, allowing 
no access by other users. Not compatible with 
other tocks. 

Places a partial lock on objects, allovtHng 
read-only access 1^ other users. Compatible 
with other write tocks and prevent full 
k)cks. 

Prevents other users from starting operations 
that require either full or wiriit© locks. 
Com^tibl® with other prevent write locks and 
prevent full locks. 

Prevents other users from starting operations 
that require full locks. Compatible with all 
locks except full locks. 
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(54) System and methods for file management 

(57) A computer system (100) has concurrently 
shared objects or resources (333) and includes a mul- 
ti-user database management system (150) having in- 
formation tables (161. 162, 163) and related objects 
stored in shared directories on a file server (180). A plu- 
rality of lock types, including directory lock, full lock, write 
lock, prevent full lock, and prevent write tock. are provkS- 
ed for controlling concurrent access. 

Methods (700) are described for managing locks by 
creating a special lock file (350) for each shared directory 
that is accessed. The lock file stores at least one logical 
lock file (400) having locking or concurrency information 
specific to a family of related members. The logical lock 
file itself includes a plurality of entries (430) for specifying 
concurrency informatksn of associated family members. 
A shared object or resource Is accessed according to the 
Information retrieved from the corresponding logical lock 
file, data retrieval being improved by reading an amount 
of data equal to a previously memorized file size plus a 
predictive amount. 
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